Statistical Mechanics of Mutual Learning with a Latent Teacher

We propose a mutual learning with a latent teacher within the framework of on-line learning, and have analyzed its dynamical behavior through the statistical mechanics method. The proposed model consists of two learning steps: two students independently learn from a teacher, and then the students learn from each other through the mutual learning. A teacher is not used in the mutual learning, so we refer to the teacher as a latent teacher. Linear perceptrons are used as the teacher and student. Our analytical results show that the overlaps between teacher and students become larger through mutual learning. In addition, we found that the mutual learning converges into the bagging of the ensemble learning scheme. We also show that a student with a larger initial overlap for mutual learning transiently passes through a state of parallel boosting in the ensemble learning in the slow learning rate limit. We have concluded that mutual learning is able to mimic the integration mechanism of bagging and parallel boosting.